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FOREWDRi^ 


The  U.S.  Army  Research  Institute  for  the  Behavioral  and  Social  Sciences 
Fort  Leavenworth  Field  Unit  conducts  a  systems  and  tralni*  g  research  program 
in  support  of  the  Combined  Arms  Center. 

The  Field  Unit  at  Fort  Leavenworth  has  been  Involved  for  several  years 
in  research  to  assess  and  Improve  decision  making  in  command  groups.  Studies 
have  been  conducted  in  the  field,  the  classroom,  and  the  laboratory. 

The  research  described  in  this  paper  is  a  major  step  forward  because  it 
identifies  a  type  of  group  problem  characterized  by  a  high  degree  of  distri¬ 
bution  of  task  responsibility.  Distribution  of  responsibility  is  typical  of 
the  decision  making  of  command  groups,  yet  virtually  none  of  the  psychological 
research  in  group  decision  making  addresses  this  problem-solving  situation. 
This  paper  also  describes  the  first  experimental  use  of  the  VARWARS  problem, 
which  provides  an  effective  tool  for  objectively  and  reliably  measuring  group 
performance. 

This  effort  was  performed  with  the  support  of  the  Combined  Arms  and 
Services  Staff  School. 


ill 


Aeeesslon  For 

BTIS  GRA4I 
DTIC  TAB 

Unanno’inced 

Justification- 


n 

□ 


By - 

Distribution/ 


Availability  Codes 
Avail  and/or 
Special 


EXPERIMENTAL  ASSESSMENT  OF  PROBLEM  SOLVING  AT  THE  CCMBINED  ARMS  AND  SERVICES 
STAFF  SCHOOL 


EXECUTIVE  SUMMARY 


Requirement : 

In  the  decision-making  procedure  of  military  command  groups,  an  overall 
problem  is  divided  into  subproblems,  and  Individuals  or  subgroups  work  to 
produce  solutions  in  their  functional  areas  of  expertise.  Eventually  the 
solutions  of  the  subgroups  are  combined  to  produce  an  overall  plan  of  action 
or  solution  to  the  problem.  This  method  of  problem  solving  can  be  difficult 
because  the  solvers  are  working  in  parallel  on  separate  but  interdependent 
problems.  If  the  subsolutions  are  to  combine  in  an  integrated  and  complemen¬ 
tary  fashion,  the  solvers  must  share  information  and  coordinate  their  solu¬ 
tions  as  they  develop  them.  The  research  literature  in  the  area  of  group 
decision  making  and  problem  solving  focuses  on  groups  that,  like  a  jury,  do 
not  divide  responsibility  for  various  tasks  among  their  members.  Many  of  the 
findings  based  on  the  jury-type  groups  do  not  seem  to  be  applicable  to  the 
area  of  command  group  problem  solving. 

The  major  goal  of  the  Combined  Arms  and  Services  Staff  School  (CAS^)  is 
to  Improve  students'  problem-solving  and  communication  skills.  CAS^  asked  the 
U.S.  Army  Research  Institute  for  the  Behavioral  and  Social  Sciences  to  develop 
a  test  that  objectively  measures  group  problem-solving  abilities  and  to  use 
the  test  to  compare  the  performance  of  captains  entering  CAS^  with  the  perfor¬ 
mance  of  captains  graduating  from  CAS^.  This  project  required  the  development 
of  a  test  Instrument  that  distributed  task  responsibility.  Such  an  Instrument 
would  satisfy  both  the  specific  need  of  CAS^  and  the  general  need  for  group 
problem-solving  research  that  is  relevant  to  U.S.  Airmy  command  groups. 


Procedure : 

A  problem  called  "VARWARS"  was  developed  to  meet  the  requirement. 
VARWARS  is  a  group  planning  and  resource  allocation  exercise  set  in  the 
context  of  acquiring,  staffing,  and  planning  for  use  of  a  hypothetical 
training  device.  A  7-person  problem-solving  group  divides  into  teams  that 
work  on  interdependent  subproblems.  Each  VARWARS  solution  is  objectively 
scorable.  In  addition,  the  problem-solving  process  is  evaluated  using  a  set 
of  scales  developed  to  measure  decision  making,  organizational  ability, 
information  sharing,  professionalism,  and  leadership. 

In  the  first  experiment,  groups  of  students  entering  CAS^  and  groups 
graduating  from  CAS^  participated  in  VARWARS  exercises.  In  the  second 
experiment,  groups  midway  through  instruction  participated.  In  the  third 
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experinent,  groups  composed  of  students  from  the  same  class  section  were  com¬ 
pared  with  groups  composed  of  students  who  had  not  worked  together  while  at 
CAS^ 


Findings : 

Students  showed  a  progressive  and  significant  decline  in  VARWARS  scores. 
Intact  groups  (from  the  same  class  section)  performed  significantly  worse  than 
groups  composed  of  students  from  different  sections.  The  process  variables 
studied  did  not  show  a  strong  relationship  to  the  VARWARS  product  scores. 

Both  graduate  and  entrant  groups  used  similar  decision-making  processes,  but 
the  graduate  groups  were  much  more  prone  to  mistakes  in  Implementation.  The 
VARWARS  score  appears  to  be  a  sensitive,  reliable  measure  of  group 
performance . 


Utilization  of  Findings: 

The  counterintuitive  results  of  this  study  have  raised  questions  that 
remain  to  be  resolved.  Basic  principles  concerning  effective  problem  solving 
in  groups  with  distributed  responsibility  have  yet  to  be  developed.  The 
VARWARS  exercise  is  a  good  tool  for  continuing  the  investigation.  CAS^  has 
made  a  major  effort  to  expand  and  strengthen  its  problem-solving  Instruction 
and  has  Included  the  VARWARS  exercise  as  a  regular  part  of  its  curriculum. 
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EXPERIMENTAL  ASSESSMENT  OF  PROBLEM  SOLVING  AT  THE  COMBINED 
ARMS  AND  SERVICES  STAFF  SCHOOL 

Introduction 


Overview 


In  a  typical  group  problem  solving  experiment,  the  members  consider  the 
problem,  generate  ideas,  and  may  propose  several  alternative  solutions.  Even¬ 
tually  a  group  solution  is  reached.  Delphi  groups  and  juries  are  examples. 
Researchers  studying  this  type  of  problem  solving  may  be  interested  in  how  the 
group  solutions  differ  in  character  or  quality  from  solutions  produced  by  an 
individual,  in  the  decision  nuking  process  of  solution  development  and  selec¬ 
tion,  or  in  the  factors  that  Influence  a  given  member’s  persuasiveness. 

This  paper  describes  research  involving  a  form  of  group  problem  solving 
that  differs  significantly  from  the  more  commonly  studied  situation  described 
above.  In  this  second  general  class  of  group  problem  solving,  the  members  do 
not  all  work  on  the  same  problem;  rather,  the  problem  is  divided  into  subprob¬ 
lems  and  the  group  members  work  to  produce  subsolutions  that  must  be  inte¬ 
grated  into  a  whole  solution.  In  the  most  difficult  of  these  cases,  the  sub¬ 
problems  are  highly  interdependent.  An  individual  subsolution  cannot  be  evalu¬ 
ated  as  good  or  bad  in  isolation;  it  is  only  effective  insofar  as  it  comple¬ 
ments  and  supports  the  other  subsolutions.  Thus  the  group  members  must  work  on 
their  problems  in  parallel  but  require  knowledge  of  the  other  members'  solu¬ 
tions.  This  situation  may  be  further  complicated  when  some  resources  are  owned 
by  the  group  as  a  whole.  Each  subgroup  cannot  be  exactly  sure  what  resources 
it  will  have  to  solve  its  subproblem  because  this  will  depend  on  the  resources 
consumed  by  the  other  subsolutions. 

A  problem  called  VARWARS,  which  requires  a  high  degree  of  distribution  of 
task  responsibility,  was  developed  for  this  research.  VARWARS  is  complex 
enough  to  allow  many  plausible  approaches  to  solutions  and  yet  is  sufficiently 
well  defined  to  enable  objective  and  quantitative  scoring  of  the  quality  of 
the  solutions.  The  VARWARS  problem  was  administered  to  groups  of  students 
entering  and  graduating  from  the  nine-week  course  of  the  Combined  Arms  and 
Services  Staff  School  (CAS"^).  Major  goals  of  the  course  are  to  Improve  the 
problem  solving  and  communication  skills  of  its  students.  The  goals  of  this 
research  were  to  experimentally  assess  student  progress  in  problem  solving  and 
to  develop  a  method  for  evaluating  group  problem  solving  skills. 

In  addition  to  the  overall  measure  of  solution  quality,  group  process  was 
evaluated  using  a  set  of  scales  developed  to  measure  organizational  ability, 
information  sharing,  decision  making,  professionalism,  and  leadership.  A  team 
of  observers  rated  groups  on  the  process  variables  in  order  to  Identify  areas 
of  weakness  in  the  students  and  to  investigate  the  relationship  between  process 
and  product. 
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The  experiment  was  designed  to  answer  three  main  questions: 

(1)  Are  graduates  of  CAS  more  effective  problem  solvers  than  entrants? 

(2)  Do  graduates  use  the  problem  solving  techniques  they  are  being  taught? 

(3)  Are  the  techniques  themselves  effective? 

Group  Problem  Solving 

There  are  important  differences  between  the  two  types  of  group  problem 
solving  situation  described  above.  In  many  cases,  factors  may  have  opposite 
influence.  In  the  jury-type  situation  groups  tend  to  be  slower  than  individu¬ 
als.  In  simple  reasoning  (Klugman,  1944),  complex  problem  solving  (Fox  and 
Lorge,  1962),  and  brainstorming  (Taylor,  Berry,  and  Block,  1958)  groups  were 
less  productive  than  individuals.  Although  not  all  studies  have  demonstrated 
this  effect,  it  is  reasonable  that  increasing  group  size  will  in  general  slow 
problem  solving.  Time  spent  explaining  solutions,  discussing  methods,  polling 
opinions,  or  dispersing  information  may  increase  solution  quality  but  reduce 
output.  This  might  be  particularly  true  in  tasks  where  one  or  two  key  indi¬ 
viduals  are  doing  the  actual  solving  or  decision  making  and  the  rest  of  the 
group  are  assisting  or  observing.  The  group  may  also  be  slowed  significantly 
when  consensus  or  general  group  approval  is  required.  Groups  working  in  the 
VARWARS-type  situation,  where  the  problem  is  subdivided,  can,  in  contrast, 
often  perform  ouch  faster  than  individuals.  Group  members  or  subgroups  can 
work  different  aspects  of  the  problem  independently  and  simultaneously.  Both 
types  of  groups  will  tend  to  be  outperformed  by  individuals  if  the  comparison 
is  done  in  person-hours  because  in  both  groups  an  amount  of  effort  must  be 
given  to  coordinating  and  communicating  which  can  be  viewed  as  overhead  to 
actual  task  performance.  Parallel  processing  in  computing  is  a  good  analog  to 
the  VARWARS-type  situation.  Computer  tasks  can  be  performed  ouch  faster  by  N 
processors  working  cooperatively  than  by  a  single  processor,  but  the  amount  of 
coordinating  tasks  increases  with  N  and  the  time  savings  typically  fall  far 
short  of  reducing  the  task  time  to  1/Nth. 

When  quality  of  the  output  is  considered,  the  two  types  of  situations  also 
present  a  strong  contrast.  Jury-type  situations  typically  produce  more  accu¬ 
rate,  higher  quality  solutions  than  Individuals  (e.g.  Anderson  1961;  Taylor  and 
Faust,  1952;  Davis  and  Restle,  1963).  One  skilled  or  intelligent  member,  who 
is  also  persuasive,  can  greatly  increase  the  quality  of  the  group  result,  espe¬ 
cially  in  tasks  where  breakthrough  reasoning,  insight,  or  flawless  analysis  are 
important.  Where  specific  knowledge  is  important  to  the  task,  the  jury-type 
group  may  be  working  from  a  knowledge  base  that  is  essentially  the  union  of  the 
knowledge  bases  of  the  individual  members.  Therefore,  performance  at  or  above 
the  quality  level  of  the  most  proficient  member  is  not  at  all  uncommon.  Sta¬ 
tistically,  the  quality  of  this  most  proficient  member  will  increase  as  group 
size  increases.  Depending  on  the  nature  of  the  task,  problem  solving  in  the 
VARWARS-type  group  may  show  just  the  opposite  effect.  For  example,  in  the 
development  of  a  new  product  the  manufacturing  and  distribution  divisions  may 
perform  brilliantly  when  compared  to  routine  performance.  As  a  result  they  can 
produce  10%  more  output  and  locate  the  product  in  stores  around  the  country  at 
10%  less  cost  than  expected.  If,  however,  marketing  has  failed  miserably  and 
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the  product  gets  a  very  negative  image,  the  outcome  of  the  entire  project  may 
be  poor.  Like  a  machine,  functioning  at  the  level  of  its  weakest  part,  the  so¬ 
lution  quality  of  VARWARS-type  groups  often  is  limited  hy  the  least  proficient 
member  or  subgroup.  In  some  instances,  none  of  the  subgroups  may  be  doing 
their  tasks  poorly  but  the  group  output  suffers  through  lack  of  integration. 
Statistically,  the  more  members  or  subgroups  the  VARWARS-type  situations  have, 
the  weaker  will  be  the  least  proficient  member.  In  a  rough  sense,  the  quality 
of  the  efforts  of  the  individual  members  in  the  jury-type  problem  combines  in 
an  additive  or  conjunctive  fashion  with  weak  members  adding  little  or  nothing 
to  the  group  output  and  strong  members  having  the  greatest  influence.  In  the 
VARWARS-type  problem,  the  member  efforts  tend  to  combine  in  a  more  complex, 
multiplicative,  or  disjunctive  fashion  with  the  weakest  members  having  the 
greatest  Influence,  often  negating  the  significance  of  the  quality  of  the 
strong  member  contributions. 

The  bulk  of  the  experimental  literature  in  group  problem  solving  involves 
studies  of  jury-type  group  situations.  Kelley  and  Thibaut,  (1969),  in  the 
Handbook  of  Social  Psychology,  sum  up  their  description  of  the  performance  of 
groups  as  "slow  but  sure".  Group  problem  solving  in  VARWARS-type  situation, 
could,  however,  be  termed  as  fast  and  faulty.  We  believe  most  group  problem 
solving  situations,  outside  the  laboratory,  fall  more  into  the  VARWARS-type 
rather  than  the  jury-type  situation.  Complex  problems  in  particular  very  often 
require  so  much  expertise  that  it  is  unreasonable  to  expect  one  person  (or  one 
or  more  members  of  the  group)  to  have  (or  to  acquire)  the  ability  to  solve  the 
problem.  It  is  equally  unreasonable  to  expect  each  member  of  the  group  to  work 
through  each  aspect  of  the  problem  together.  Many  group  problem  solving  situa¬ 
tions  may,  on  the  surface,  appear  to  be  jury-type,  for  example  the  activity  of 
various  committees.  Often  however,  committee  members  will  each  have  various 
areas  of  expertise  and  responsibility.  Each  may  be  expected  to  analyze  the 
problem  from  a  particular  perspective  and  present  estimates  and  analyses  for 
the  other  members  to  weigh  in  making  the  decision.  There  are  strong  elements 
of  the  VARWARS-type  situation  here,  where  the  analyses  from  different  perspec¬ 
tives  actually  represent  subproblems  and  only  the  presenter  is  in  a  good  posi¬ 
tion  to  judge  the  quality  and  accuracy  of  his  contribution. 

The  Combined  Arms  and  Services  Staff  School 

CAS  is  a  9-week  course  now  given  to  all  U.S.  Arn^  officers  when  they  are 
approximately  one-third  of  the  way  through  their  careers.  Most  students  are 
captains  in  rank  although  some  are  majors.  The  course  is  designed  to  teach  the 
fundamental  skills  required  of  staff  officers  at  the  brigade  level  and  above. 
The  goals  of  the  course  are  to  inqirove  the  students  written  and  oral  communica¬ 
tion  and  problem  solving  abilities.  The  students  are  assigned  to  12  person 
groups  that  live  and  work  together  and  are  trained  a  single  Instructor. 

CAS^  instructors,  or  staff  leaders,  are  lieutenant  colonels  and,  for  the  most 
part,  are  a  select  group  of  officers  of  this  rank  who  have  been  especially 
successful  in  their  careers. 

Students  are  taught  to  follow  a  six-step  analytic  problem  solving  method 
composed  of  the  following  steps:  identify  the  problem,  gather  information, 
generate  alternative  solutions,  analyze  the  alternatives,  decide,  and  ii]q}lement 
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the  solution.  During  the  course,  the  students  are  given  several  decisionmaking 
exercises.  They  work  as  a  group  to  reach  a  decision,  prepare  a  briefing  to 
explain  and  justify  their  decision  and  present  the  briefing  to  their  instructor 
who  critiques  their  performance.  Kaplan  and  Klyohara,  (1985)  examined  instruc¬ 
tor  ratings  of  entering  and  graduating  students  and  found  evidence  of  improved 
briefing  style;  better  gestures,  movement,  eye  contact,  volume  and  rate  of 
delivery.  The  content  of  the  briefings,  however,  were  not  rated  as  better. 

The  VARWARS  Problem 

VARWARS  is  a  group  planning  and  resource  allocation  exercise  which  is  set 
in  the  context  of  acquiring,  staffing  and  employing  a  new,  hypothetical,  train¬ 
ing  device.  The  VARWARS  problem  was  developed  such  that: 

1.  The  quality  of  the  group  solution  is  numerically  scorable  in  an  objec¬ 
tive  manner. 

2.  The  subproblems  exhibit  considerable  dependency.  Including  sharing  of 
a  common  pool  of  resources. 

3.  There  is  not  a  single  correct  answer,  for  example,  as  in  a  math  prob¬ 
lem.  There  are  many  good  answers. 

A.  The  presentation  by  the  group  of  their  solution  is  very  simplified  so 
that  group  score  depends  on  the  quality  of  the  problem  solving  rather  than  the 
presentation  of  the  solution. 

5.  All  solutions  are  scorable,  even  "defective"  solutions. 

6.  Problem  elements  are  straightforward  so  that  group  success  depends 
more  on  the  ability  of  the  group  to  share  information,  make  decisions,  inte¬ 
grate  the  subsolutions  and  otherwise  function  effectively  than  it  does  upon 
the  creativity  or  insight  of  a  single  member. 

In  the  VARWARS  exercise,  a  leader  is  assigned,  and  the  remaining  members 
of  the  group  divide  themselves  into  three  two-person  teams:  a  Budget  Team,  a 
Training  Team,  and  a  Personnel  Team.  In  addition  to  general  instructions  re¬ 
ceived  by  all  members,  each  team  and  the  leader  receives  packets  of  individual 
instructions  and  information.  In  order  to  effectively  solve  the  problem,  each 
team  requires  information  originally  provided  to  the  other  teams  and  the 
leader.  The  teams  also  depend  on  one  another  for  intermediate  products  such  as 
estimates  and  assessments. 

Although  it  is  not  made  explicit  in  the  VARWARS  instructions,  the  VARWARS 
problem  may  be  viewed  as  a  two-phase  resource  allocation  problem.  The  group 
begins  with  a  fixed  amount  of  resources:  13  million  dollars  and  150  weeks.  In 
the  first  phase,  the  Budget  and  Personnel  Teams  convert  the  initial  resources 
into  a  potential  for  providing  VARWARS  training.  A  nxanber  of  VARWARS  machines 
are  purchased,  operators  are  hired  to  run  them,  and  money  is  set  aside  for 
maintenance  and  operating  costs.  As  a  result  of  Budget  and  Personnel  deci¬ 
sions,  the  Training  Team  is  provided  with  a  fixed  amount  of  resources;  a  num¬ 
ber  of  devices,  each  with  an  established  capacity  for  training,  and  some  money 
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budgeted  to  pay  for  travel  of  the  trainees.  In  the  second  phase  of  the  prob¬ 
lem,  the  Training  Team  must  convert  these  resources  to  points  by  locating  the 
devices  at  military  installations  and  scheduling  units  for  training  on  the 
device.  Work  on  both  phases,  however,  proceeds  concurrently. 

Although  the  number  of  the  variables  involved  makes  the  VARWARS  problem 
complex,  the  scoring  concept  is  relatively  simple.  The  group  converts  a  fixed 
amount  of  resources  into  points.  The  more  efficiently  they  perform  this  opera¬ 
tion  the  more  points  they  will  score.  The  VARWARS  score  's  based  on  the  point 
system  described  in  the  Leader's  Instructions.  There  are  88  military  units 
eligible  for  training  in  the  problem,  composing  five  different  types  of  unit: 
active  corps,  active  divisions,  active  brigades,  reserve  divisions,  and  reserve 
brigades;  points  are  awarded  for  each  unit  in  accordance  with  the  number  of 
persons  trained  from  that  unit  and  in  accordance  with  the  type  of  unit.  The 
analysis  and  scoring  of  VARWARS  solutions  is  done  by  a  computer  program. 

The  gproup  score  depends,  of  course,  on  the  quality  of  the  decisions  made, 
e.g. ,  which  model  devices  are  purchased  or  where  the  devices  are  located. 

Also,  the  group  score  depends  greatly  on  the  amount  of  resources  that  are 
wasted  by  misunderstandings  and  poor  decisions,  e.g.,  the  Training  Team  uses 
only  a  small  fraction  of  the  resources  that  have  been  developed  by  the  other 
teams,  or  the  Personnel  Team  consumes  such  a  large  portion  of  the  total  budget 
that  the  other  teams  have  only  enough  resources  to  use  a  small  fraction  of  the 
personnel  hired. 

The  score  the  group  receives  is  clearly  not  a  reflection  of  the  sum  of  the 
quality  of  the  individual  team  products.  The  team  products  are  highly  interde¬ 
pendent.  If  one  team  produces  a  product  which  is  not  well  Integrated  with  the 
other  team  products,  then  the  group  score  may  be  drastically  reduced.  There¬ 
fore,  the  group  score  is  more  similar  to  a  nultlpllcatlve  function  of  the  indi¬ 
vidual  team  products  than  it  is  to  an  additive  function. 

A  detailed  description  of  the  VARWARS  Problem  may  be  found  in  Varwars !  A 
Group  Problem  Solving  Exercise  (Lussler,  1988). 


Experiment  1  Methods 

Participants 

Twenty-two  problem  solving  groups,  consisting  of  seven  members  each,  par¬ 
ticipated  in  the  experiment.  Group  members  were  U.S.  Arn^  Captains  attending 
one  of  three  rotations  at  CAS"^.  Eleven  of  the  groups,  the  entrant  groups,  were 
composed  of  captains  randomly  selected  during  registration  for  the  course.  The 
entrant  groups  were  tested  during  one  of  the  two  weekdays  immediately  preced¬ 
ing  the  first  day  of  instruction.  The  other  11  groups,  the  graduate  groups, 
were  composed  of  officers  in  the  seventh  week  of  the  nine-week  course.  Members 
of  each  graduate  group  were  drawn  from  different  class  sections,  so  members  had 
not  worked  together  while  at  the  course.  Graduate  group  members  were  selected 
to  match  entrant  group  members  on  branch  of  service,  and  on  a  diagnostic  test 
of  quantitative  skills  given  routinely  to  all  students. 
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Procedures 


Groups  were  tested  at  the  A.RI  Fort  Leavenworth  small  group  laboratory 
which  provides  videotape  recording  and  observation  of  the  group  through  a  one¬ 
way  mirror*  Each  session  commenced  with  a  brief  introduction  by  a  researcher. 
During  the  introduction,  the  senior  member  of  the  group  was  assigned  the  posi¬ 
tion  of  group  leader,  equipment  and  materials  available  were  identified,  and 
the  written  instructions  to  the  VAKWARS  problem  were  distributed*  Verbal  in¬ 
structions  emphasized  the  three  hour  time  limit,  the  requirement  to  solve  the 
problem  using  only  the  equipment  and  materials  provided,  and  the  fact  that  the 
group's  score  on  the  exercise  depended  solely  on  the  solution  provided  on  their 
data  forms,  i*e*,  no  written  nor  oral  justification  nor  briefing  of  their  solu¬ 
tion  would  be  required*  During  the  exercise,  three  researchers  observed  the 
group  from  an  adjoining  observation  room* 

Variables 

3 

The  only  independent  variable  was  state  of  training:  before  or  after  CAS 
instruction*  Dependent  variables  consisted  of  an  outcome  score  and  process 
ratings*  The  outcome  score,  or  product  score,  is  a  single  objective  measure  of 
the  quality  of  a  group's  solution  (Lussier,  1988)*  The  process  ratings  repre¬ 
sent  a  consensus  rating  provided  by  the  three  authors  using  rating  scales  re¬ 
flecting  the  processes  used  in  arriving  at  the  solution*  Five  process  scales, 
each  allowing  a  maximum  of  20  points,  were  used*  The  first.  Getting  Organized, 
rated  groups  on  their  organization  into  teams,  on  their  time  management  plan, 
on  their  problem  definition  and  restatement,  and  on  their  preliminary  informa¬ 
tion  gathering*  The  second  process  scale.  Information  Sharing  and  Coordination, 
contained  twenty  items  of  information  which  were  available  to  one  team  but  were 
of  value  to  another  team  in  their  decision  making*  Observers  noted  each  in¬ 
stance  where  they  believed  one  of  these  twenty  information  items  was  shared 
with  the  appropriate  team.  Some  of  the  information  items  were  facts  provided  in 
the  instructions*  Other  items  were  estimates  which  needed  to  be  produced  by 
one  team  to  aid  another*  The  third  process  scale.  Decision  Making,  rated 
groups  on  how  well  they  developed  courses  of  action,  generated  criteria  for 
selection  of  a  course  of  action,  analyzed  courses  of  action,  prepared  esti¬ 
mates,  and  selected  a  course  of  action*  The  fourth  process  scale.  Profession¬ 
alism,  rated  groups  on  their  dedication  to  task,  error  checking,  on  whether 
they  performed  an  Incremental  Improvement  of  a  complete  solution,  and  on  their 
ability  to  work  within  deadlines  and  manage  time*  The  last  scale.  Leadership, 
rated  the  assigned  leader  on  providing  focus  and  direction,  making  and  stating 
decisions  clearly,  conflict  resolution,  and  keeping  the  group  to  an  appropriate 
time  schedule. 

The  three  observers  rated  the  groups  independently  during  the  exercise  and 
afterward  were  required  to  reach  consensus  on  each  item*  Further  details  of 
the  process  evaluation  procedure  may  be  found  in  Garllnger  and  Lussier,  (1988). 

The  product  score,  a  measure  of  the  quality  of  the  group  solution,  primar¬ 
ily  reflects  the  efficiency  of  the  solution,  i*e* ,  how  many  points  can  be 
achieved  given  the  fixed  resources  (time  and  money)  of  the  problem*  Because 
this  is  the  first  experiment  using  the  VARWARS  problem,  normative  data  regard¬ 
ing  scores  are  not  available*  The  optimal  solution  is  difficult  to  determine 
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exactly  but  appears  to  be  close  to  but  below  a  score  of  800*  A  score  of  700  or 
above  would  appear  to  be  a  practical  maximum  In  a  time  limited  situation* 

Scores  in  the  range  of  500-600  are  good  workmanlike  solutions*  A  group  with 
scores  In  this  range  has  probably  made  a  large  number  of  small  errors  but  it 
has  made  reasonable  decisions  and  Implemented  them  without  major  error.  Scores 
below  450  almost  always  involve  at  least  one  siajor  error* 


Experiment  1  Results 


Product  Scores 

Scores  for  the  22  groups  are  shown  in  Figure  1*  The  mean  score  for  the  11 
entrant  groups  was  465*3  with  a  standard  error  of  25*8  points*  The  mean  score 
for  the  graduate  groups  was  356«4  with  a  standard  error  of  34*4  points*  As  is 
clear  from  Figure  1,  the  entrants  scored  significantly  higher  (paired  t  * 

2.644,  p<.025)  than  the  graduates*  The  relatively  small  amount  of  overlap 
between  the  distributions  is  remarkable*  It  is  also  notable  that  the  109  point 
difference  between  the  groups  was  not  in  the  expected  direction* 
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of 
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Figure  1.  Distribution  of  VARWARS  product  scores 
for  graduates  and  entrants. 


Process  Scores 

Table  1  shows  the  mean  process  scores  for  the  two  types  of  group*  While 
the  entrants  exceeded  the  graduates  on  all  scale  items,  the  difference  was 
significant  only  for  the  Information  Sharing  scale,  which  measured  the  transfer 
of  specific  Information  items*  The  final  column  sliows  the  reliability  coeffi¬ 
cients  for  the  three  raters,  indicating  that  there  was  good  agreement  on  the 
process  ratings* 
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Table  I 


Mean  Process  Scores  for  Entrants  and  Graduates 


Scale 

Entrants 

(N-11) 

Graduates 

(N-11) 

Difference 

Rater 

Reliability 

1. 

Getting  Organized 

13.8 

13.2 

0.6 

.91 

2. 

Information  Sharing 

15.8 

13.3 

2.5** 

.86 

3. 

Decision  Making 

12.5 

12.4 

0.2 

.70 

4. 

Professionalism 

12.4 

10.1 

2.3 

.95 

5. 

Leadership 

13.6 

12.5 

l.i 

.95 

Total 

68.1 

61.5 

6.7 

Standard  Error 

4.6 

3.8 

6.2 

**  p  <  .01. 


One  item  meriting  closer  examination  is  the  "Dedication  to  Task"  item  on 
the  Professionalism  scale.  This  item  might  indicate  whether  the  graduates, 
after  2  months  of  class,  had  experienced  a  loss  of  motivation  compared  to  en¬ 
tering  students,  which  could  account  for  the  performance  decrement*  Table  2 
shows  the  number  of  groups  rated  in  each  category  on  the  "Dedication  to  Task" 
item. 


Table  2 


Distribution  of  Observer  Ratings  on  "Dedication  to  Task" 


Category 


Entrants _ Graduates 


1.  Excessive  amounts  of  Irrelevant  2 

behaviors  or  Idleness  seriously 

impaired  group  performance. 

2.  Moderate  amounts  of  irrelevant  1 

behaviors  or  Idleness  had  some 

negative  impact  on  performance. 

3.  Some  members  failed  to  maintain  0 

attention  to  task  despite 

prodding. 

A.  All  team  members  worked  2 

continuously  but  some  required 
prodding  by  others. 

5.  Entire  group  showed  enthusiasm  6 

and  attention  to  task  throughout. 


0 


2 


1 


7 


1 


The  difference  is  not  significant,  nor  did  the  raters  subjectively  notice 
an  obvious  difference  in  motivation.  Nonetheless,  Table  2  does  admit  the  pos¬ 
sibility  that  the  groups  differed  at  the  highest  level,  with  most  of  the  en¬ 
trant  groups  rated  as  five  and  most  of  the  graduate  groups  rated  as  four. 

Another  item  of  importance  is  "Develop  Courses  of  Action",  because  it  is 
an  essential  part  of  the  method  taught  in  the  course,  l.e.,  develop  several 
good  distinct  alternative  solutions,  establish  criteria  for  comparison,  analyze 
each  alternative,  tuid  select  the  best.  Table  3  shows  the  ratings  by  the  ob¬ 
servers. 
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Table  3 


Dlstrlbutloa  of  Observer  Ratings  on  "Develop  Q>urses  of  Action" 


Category 


Entrants _ Graduates 


1.  Group  did  not  arrive  at  any  overall  1 

course  of  action  but  worked  on 

completely  unintegrated  subsolutions* 

2.  Proposed  only  one  course  of  action  1 

which  was  accepted  with  insufficient 

analysis. 

3.  Course  of  action  generation  proceeded  8 

sequentially.  A  single  course  of 

action  was  analyzed  for  feasibility 
before  alternatives  were  considered. 

4.  Proposed  two  or  more  courses  of  1 

action  that  were  not  sufficiently 

distinct  or  were  inadequate  in 
concept  OR  group  was  inefficient 
and  used  excessive  time  in 
generating  courses  of  action. 

5.  Proposed  two  or  more  adequate  and  0 

distinct  courses  of  action. 


0 


4 


5 


1 


1 


Neither  entrants  nor  graduates  followed  the  method  taught,  as  only  one 
group  proposed  more  than  one  adequate  alternative,  and  that  group  failed  to 
complete  the  process  by  analyzing  both  alternatives.  The  primary  process  of 
problem  solving  employed  by  both  types  of  groups  was  to  develop  a  single  solu~ 
tlon,  modifying  it  as  necessary  and  rejecting  it  only  if  it  became  coDq>letely 
unworkable*  The  graduates  appeared  less  likely  to  complete  a  sufficient  analy~ 
sis,  although  the  difference  was  not  significant. 

Relationship  between  Process  and  Product  Scores 

When  all  22  groups  are  combined,  there  was  no  significant  relationship 
between  product  score  and  overall  process  score,  nor  between  product  score  and 
any  of  the  five  scale  scores  that  compose  the  process  score*  Table  4  shows 
correlations  between  the  five  process  scales  and  product  scores  when  the  en¬ 
trant  and  graduate  groups  are  considered  separately.  Each  correlation  is  based 
on  11  cases. 
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Table  4 


Correlations  between  Process  Scales  and  Product  Score 


Process  Scale 

Entrants 

Graduates 

Getting  Organized 

-.44 

-.08 

Information  Sharing 

-.40 

.24 

Decision  Making 

-.15 

.61* 

Professionalism 

-.28 

.47 

Leadership 

-.34 

.30 

Total  Process  Score 

-.39 

.34 

*  p  <  .05. 

It  is  doubtful  that  the  greater  product  scores  found  in  the  entrants  when 
compared  to  graduates  can  be  attributed  to  their  greater  Information  Sharing 
scores  because  the  correlation  between  Information  Sharing  and  product  score 
for  the  entrants  is  negative  and  nonsignificant.  The  only  correlation  reaching 
significance  is  between  product  score  and  Decision  Making  scale  for  the  gradu¬ 
ate  group.  The  four  items  which  compose  this  scale  and  their  correlations 
with  product  score  are  shown  in  Table  5. 

Table  5 

Correlations  between  Decision  Making 
Graduate  Groups 

scale  items 

and  product  scores  for  11 

Items  of  Decision  Making  Scale 

Correlation 

with  Product  Score 

Develop  Courses  of  Action 

.05 

Generate  Criteria  for  Selection 

.56* 

Analyze  Courses  of  Action  and 

Prepare  Estimates 

.69* 

Selection  of  Course  of  Action 

.39 

*  p  <  .05. 
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Tables  6  and  7  show  the  categories  for  the  two  significant  items.  For 
each  category  the  number  of  graduate  groups  rated  in  that  category  is  shown, 
along  with  the  product  scores  for  these  groups. 


Table  6 

Product  scores  of  graduate  groups  rated  in  each  category  of  "Generate  Criteria 
for  Selection"  item 


Category 


Number  of 
Graduate  Groups 


1.  No  criteria  established.  Group  0 

seemed  unaware  of  need  for  criteria. 

2.  Discussed  need  for  criteria  but  1 

none  established. 

3.  Criteria  were  proposed  but  group  4 

failed  to  reach  consensus  or  achieve 

a  common  understanding  of  the 
criteria. 

4.  At  least  one  relevant  criteria  was  5 

proposed  but  clearly  relevant 

criteria  were  Ignored  OR  process 
was  inefficient  and  overly  time 
consuming. 

5.  Criteria  were  relevant,  clearly  de-  1 

fined  and  understood,  and  no  clearly 

relevant  criteria  were  Ignored. 


VAKWARS 
Produc  t 
Scores 


388 


98,  229, 
337,  361 


348,  366, 
398,  423, 
431 


541 
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Table  7 


Product  scores  of  graduate  groups  rated  in  each  category  of  "Analyze  Courses  of 
Action  and  Prepare  Estimates"  item 


Category 

Number  of 

Graduate  Groups 

VARWAkS 

Product  Scores 

I. 

No  estimate  made  or  estimates 
based  on  meager  analysis. 

0 

2. 

All  teams  begin  work  but  none 
complete  a  satisfactory 
analysis. 

4 

98,  229,  337 

3. 

Only  one  team  analyzed  COAs 
sufficiently. 

4 

348,  366,  388, 
398 

4. 

Two  teams  analyzed  COAs 
sufficiently. 

2 

423,  541 

5. 

Each  COA  was  analyzed  completely 
completely  by  all  teams.  Costs 
and  benefits  were  accurately 
identified. 

1 

431 

These  two  items,  in  a  sense,  represent  intermediate  team  products  more 
than  process  variables.  The  "Criteria"  item  involves  the  determination  of  what 
analyses  or  estimates  need  to  be  made  while  the  "Analyze"  item  measures  how 
many  teams  complete  the  analysis  satisfactorily.  It  is  therefore  not  surpris¬ 
ing  that  these  variables  were  correlated  with  the  product  score.  More  surpris¬ 
ing  possibly  is  that  these  items  were  not  significantly  correlated  (-.16  for 
"Criteria"  and  .12  for  "Analyze")  for  the  entrants.  The  entrants  and  gradu¬ 
ates,  however,  were  rather  distinct  groups.  Almost  all  of  the  graduate  solu¬ 
tions  contained  major  errors,  and  some  made  glaring  blunders,  while  most  of  the 
entrant  solutions  could  be  characterized  as  having  several  minor  errors  and 
only  a  few  had  major  errors.  Possibly  the  greater  errors  of  the  graduates  may 
have  been  more  evident  to  the  observers.  Perhaps  the  failure  to  establish 
appropriate  criteria  and  to  perform  adequate  analyses  may  be  more  significant 
determinants  of  final  score  among  groups  whose  final  solutions  are  more  er¬ 
ror-ridden. 


Experiment  2  Methods 

In  Experiment  1,  the  graduate  groups  were  composed  of  students  in  the 
seventh  week  of  the  nine  week  course  because  it  was  felt  that  students  in  the 
eighth  and  ninth  weeks  might  be  showing  a  motivational  "letdown"  due  to  the 
approaching  end  of  their  assignment.  In  this  experiment,  6  groups  composed  of 
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students  in  the  3rd  week  of  CAS*^  training  were  tested.  Students  at  this  point 
had  coBq>leted  the  problem  solving  block  of  Instruction  although  they  had  only 
participated  in  about  half  as  many  group  exercises  as  the  graduate  group  of  Ex¬ 
periment  1.  As  in  Experiment  1,  the  seven  members  of  each  "mid-course"  group 
are. drawn  from  separate  classes  so  that  the  members  had  not  worked  together  at 


CAS- 


Testing  was  conducted  in  the  same  manner  as  in  Experiment  1. 


Experiment  2  Results 

The  six  midcourse  groups  had  a  mean  product  score  of  440.3  points  with  a 
standard  error  ot  48.2  points.  It  is  difficult  to  draw  conclusions  based  on 
such  a  relatively  small  number  of  groups.  The  midcourse  results  are  consistent 
with  the  previous  results  in  that  the  average  product  score  fell  between  the 
entrants  and  graduates  and  was  not  significantly  different  from  either.  The 
mean  process  score  for  the  6  midcourse  groups  was  61.2  with  a  standard  error  of 
4.83  which  was  not  significantly  different  from  either  the  entrants  or  gradu¬ 
ates. 


Experiment  3  Methods 

In  Experiment  3,  12  groups  of  CAS  students  participated  in  VARWARS  exer¬ 
cises.  Testing  was  conducted  as  in  Experiments  1  and  2  with  the  following  four 
differences.  Testing  was  conducted  during  Week  Six  of  the  course.  The  groups 
were  not  observed  by  process  raters.  Groups  were  tested  in  their  own  classroom 
rather  than  at  the  ARl  laboratory  facility.  Finally,  six  of  the  groups,  the 
"intact"  groups,  were  each  composed  of  seven  students  from  the  same  class. 
Thus,  members  of  the  intact  groups  had  been  working  together  on  problem  solving 
teams  for  the  previous  six  weeks.  The  other  six  groups,  the  "mixed"  groups, 
were  each  composed  of  seven  students  from  seven  different  classes  as  in  Experi¬ 
ments  1  and  2. 


Experiment  3  Results 

Mixed  groups  produced  solutions  with  a  mean  score  of  410.5  and  standard 
error  of  22.3.  Intact  groups  produced  solutions  with  a  mean  score  of  285.5  and 
a  standard  error  of  53.5.  Despite  the  small  number  of  groups,  the  Intact  group 
scores  were  significantly  worse  (t~2.156,  p<.05)  than  the  mixed  groups. 

It  is  not  possible  to  assess  the  effects  of  the  move  from  the  laboratory 
to  classroom  testing;  however,  the  mean  score  of  the  mixed  groups  is  consistent 
with  the  findings  of  the  previous  experiments.  Figure  2  shows  the  mean  scores 
of  all  five  groups  in  Experiments  1,  2,  and  3  plotted  as  a  function  of  days  of 
CAS^  training. 
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DAYS  OF  INSTRUCTION 


Figure  2.  Means  and  standard  errors  of  the  mean 

for  all  groups  tested  In  Experiments 
1>  2,  and  3. 


which  they  had  been  tau^t  SL  *  Vtoblnm  solving  techniques 

efficacy  of  the  problem  solving  techniquS* ‘d* 
because  no  groups  used  the  techniques.  ^  ^  •ddresaed 
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Lack  of  Explanation  of  Significant  Findings 


The  process  variables  measured  do  not  help  to  explain  the  clear  differ- 
ences  in  performance  between  the  graduates  and  entrants  because  the  process 
variables  neither  appear  to  contribute  greatly  to  the  product  score  nor  to 
differ  between  the  entrants  and  graduates.  It  is  Important  to  realize  that 
most  of  the  solutions  by  the  graduates  contained  major  flaws,  while  few  of  the 
entrant  solutions  did.  Therefore,  the  important  characteristic  is  not  a  high 
level  problem  solving  ability  of  the  entrants.  Their  solutions  were  workmanlike 
at  best,  taking  one  of  a  great  many  of  reasonable  solutions  and  implementing  it 
with  only  minor  errors.  The  significant  effect  resulted  from  the  mistake-rid¬ 
den  behavior  of  the  graduates.  Moreover,  in  most  of  the  graduate  exercises, 
one  could  not  blame  a  single  Individual.  Since  the  group  was  divided  into  two 
person  teams,  there  was  always  one  person  watching  and  assisting  a  mistake 
maker  and  usually  there  were  several  people  on  other  teams  who  had  the  opportu¬ 
nity  to  detect  the  error.  Often  the  mistakes  were  made  the  group  as  a 
whole. 


The  fairly  obvious  possibility  is  that  after  several  weeks  in  the  course, 
student  motivation  begins  to  flag  and  the  students  put  forth  less  effort  and 
were  more  careless,  resulting  in  flawed  solutions.  This  explanation,  however, 
does  not  indicate  why  the  intact  groups  scored  worse  than  mixed  groups  at  the 
same  point  in  the  course.  Further,  a  loss  of  motivation  ms  not  apparent  to 
the  observers.  Considering  the  observer  ratings  on  the  "Enthusiasm  and  Dedica¬ 
tion  to  Task"  item,  it  seems  that  if  there  was  a  difference  in  motivation  then 
1)  the  difference  was  small,  2)  it  affected  only  one  or  two  members  of  the 
group,  and  3)  it  was  a  difference  between  good  motivation  and  very  good  motiva¬ 
tion  rather  than  Involving  low  levels  of  motivation. 

It  seems  probable  that  patterns  of  behavior,  which  are  harmful  to  group 
problem  solving,  develop  during  the  course.  One  possibility  is  that  the  group 
members,  who  work,  play,  and  live  together  for  nine  weeks  come  to  place  high 
value  on  harmonious  relations,  and  become  less  critical  of  one  another's  ef¬ 
forts.  Additionally,  punishments  in  class  are  almost  always  directed  at  an 
individual  -  if  an  individual's  work  is  below  standard  then  he  or  she  must  redo 
it.  Students  may  adopt  an  "I'll  do  ny  job,  you  do  your  job"  attitude  which 
inhibits  satisfactory  error-checking  among  the  group. 

Another  possible  explanation  for  the  lower  scores  of  the  more  advanced 
groups  is  that  they  are  mistakenly  emphasizing  the  wrong  aspects  of  the  prob¬ 
lem.  The  problems  students  are  trained  on  tend  to  begin  with  a  presentation  of 
several  good  close  alternatives.  The  students  are  then  tasked  to  choose  one  and 
present  a  cogent  defense  of  their  selection.  Each  of  the  alternative  solutions 
has  merits  (or  the  problem  would  be  easy)  so  the  students  may  learn  that  it  is 
not  so  important  which  solution  is  chosen  and  that  the  greatest  effort  needs  to 
go  into  justifying  the  decision.  In  VARWARS,  however,  the  score  depends  on 
choosing  a  good  solution. 

Several  other  explanations  are  also  possible.  However,  whatever  may  be 
the  cause  of  the  effect,  it  seems  evident  that  forming  mixed  groups  of  students 
only  partially  disrupted  the  tendency  of  graduates  to  make  major  mistakes. 
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Relationship  between  Process  and  Product 


In  general,  the  process  items  which  related  to  high  product  scores  were 
those  which  were  closely  tied  to  the  problem  and  to  mistakes  In  intermediate 
products,  i.e* ,  poor  or  erroneous  estimates,  neglected  criteria,  or  other  major 
errors.  Pure  process  items  addressing,  for  example,  whether  the  group  restated 
and  redefined  the  problem,  whether  member  qualifications  were  considered  when 
the  problem  was  subdivided,  whether  there  was  a  clear  time  management  plan, 
whether  distinct  courses  of  action  were  proposed,  whether  the  leader  made  and 
stated  decisions  clearly,  etc.  ,  did  not  appear  to  relate  to  group  success  in 
our  experiments. 

The  lack  of  relationship  between  some  of  the  process  variables  and  product 
quality  may  be  due  to  a  lack  of  variability  in  our  sample  rather  than  to  the 
lack  of  an  actual  relationship  between  the  variables.  Most  groups  tested  in 
our  sample  performed  poorly  in  Identifying  subproblems  and  potential  solution 
procedures,  in  developing  alternate  courses  of  action,  and  in  time  management. 
Members  rarely  used  rough  estimation  correctly  to  scope  the  problem  and  to 
prune  unpromising  branches  of  investigation  in  a  timely  manner.  Typically, 
queries  for  Information  were  either  answered  Immediately  in  too  vague  or  gen¬ 
eral  a  manner  or  were  answered  too  late  in  too  nich  detail.  Nonetheless,  our 
evidence  suggests  that  quality  of  the  problem  solving  products  in  a  complex 
situation  are  not  clearly  tied  to  the  problem  solving  process. 

The  VARNARS  Problem 


One  of  our  goals  was  to  devise  an  exercise  featuring  interdependent  sub¬ 
groups  and  objective  scoring.  VARWARS  worked  well  because  it  is  complex  enough 
so  that,  at  least  with  a  3-hour  solving  period,  no  single  member  could  absorb 
all  the  details  and  make  all  the  decisions.  The  problem  has  to  be  worked  by 
teams  in  parallel.  Further,  the  complexity  sets  an  environment  of  confusion  and 
potential  error  in  which  communication  between  group  members  becomes  a  signifi¬ 
cant  determinant  of  success. 

Because  of  the  clear  differences  between  graduates  and  entrants,  and  be¬ 
tween  mixed  and  intact  groups,  it  seems  evident  that  product  scores  in  VARWARS 
can  discriminate  between  groups  and  are  not  unduly  affected  by  chance  factors. 

While  this  study  did  not  directly  address  the  differences  between  the 
VARWARS  problem  and  jury-type  problems,  it  seems  likely  that  VARWARS  will  be  a 
suitable  vehicle  for  conducting  such  comparisons.  The  major  impact  on  group 
score  by  the  mistakes  of  some  members  found  in  this  study  is  consistent  with 
the  proposed  distinction  between  VARWARS— type  and  jury— type  problems.  Further 
investigations  must  address  the  issue  of  the  efficacy  of  the  prescribed  problem 
solving  method,  provided  that  some  groups  can  be  induced  to  use  it.  Also,  the 
underlying  causes  of  the  decline  in  scores  of  "trained"  problem  solvers  at  CAS^ 
must  be  resolved. 


17 


REFERENCES 


Anderson,  N.  H.  (1961).  Group  performance  in  an  anagram  task.  Journal  of 
Social  Psychology.  55.  67-75. 

Davis,  J.  H. ,  &  Restle,  F.  (1963).  The  analysis  of  problems  and  prediction  of 
group  problem  solving.  Journal  of  Abnormal  and  Social  Psychology.  66. 
103-116. 

Fox,  D.  J. ,  &  Lorge,  I.  (1962).  The  relative  quality  of  decisions  vfrltten  by 
individuals  and  by  groups  as  the  available  time  for  problem  solving  is 
increased.  Journal  of  Social  Psychology.  57.  227-242. 

Garlinger,  D.  K. ,  &  Lussier,  J.  W.  (1988).  VARWARS  Exercise  Evaluation 

Procedure  Process  Rating  Scales  Users*  Guide  (ARl  Research  Product  88-16). 
Alexandria,  VA:  U.S.  Arn^  Research  Institute  for  the  Behavioral  and  Social 
Sciences.  (AD  A198  937). 

Kaplan,  1.  T. ,  &  Kiyohara,  0.  M.  (1985).  Effects  of  Training  on  Staff  Officer 
Performance  (Unpublished  manuscript).  U.S.  Arniy  Research  Institute  for  the 
Behavioral  and  Social  Sciences,  Alexandria,  VA. 

Kelley,  H.  H. ,  &  Thlbaut,  J.  W.  (1969).  Group  problem  solving.  In  G.  Lindzey 
&  E.  Aronson  (Ed.),  Handbook  of  Social  Psychology  (2nd  ed. ,  Vol.  4,  pp. 
1-101).  Reading,  Massachusetts:  Addison-Wesley. 

Klugman,  S.  F.  (1944).  Cooperative  vs.  individual  efficiency  in  problem 
solving.  Journal  of  Educational  Psychology.  Jl,  91-100. 

Lussier,  J.  W.  (1990).  VARWARS;  A  group  Problem  Solving  Exercise.  (ARI 
Research  Product  90-09).  Alexandria,  VA:  U.S.  Army  Research  Institute 
for  the  Behavioral  and  Social  Sciences.  (AD  B143  860) 

Taylor,  D.  W. ,  Berry,  P.  C.  &  Block,  C.  H.  (1958).  Does  group  participation 
when  using  brainstorming  facilitate  or  inhibit  creative  thinking. 
Administrative  Science  Quarterly,  23-47. 

Taylor,  D,  W.,  &  Faust,  W.  L.  (1952).  Twenty  questions:  efficiency  in  problem 
solving  as  a  function  of  size  of  group.  Journal  of  Experimental  Psychology. 
44,  360-368. 


920709 


19 


